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Abstract 

We compare Stochastic and Worst-case condition numbers and loss 
of precision for general computational problems. We show an upper 
bound for the ratio of Worst-case condition number to the Stochastic 
condition number of order 0{^/rn). We show an upper bound for the 
difference between the Worst-case loss of precision and the Stochastic 
loss of precision of order 0(ln m) . The results hold if the perturbations 
are measured norm- wise or componentwise. 

1 Introduction 

Let X G and f{x) G be the input and output of a computational prob- 
lem respectively. We assume f{x) is different iable. Condition numbers are 
real numbers measuring the sensitivity of the output f{x) to the input x of the 
problem. But there are many different versions of condition numbers. Below 
are the definitions of Worst-case Norm-wise, Worst-case Component- 
wise, Stochastic Norm-wise and Stochastic Component-wise condi- 
tion numbers. For any set S, we write x ~ iS if x is a random variable (or 
vector) uniformly distributed in S. And we denote by E f{x) the expected 
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value of f{x) when x ~ <S. 



WNC(a;) = hm sup "^^"^'^ ^^"^ 



■5^0 ^'ePM) (^ll/WII 
WCC,(x) = hm sup '^^'^^?~{^|^''^ 

SNC(x) = hm E M(|lz^Mand 
x'^r{x,5) 5\\f{x)\\ 

SCCAx) = hm E LM^IZ^M where 

V{x,S) = {x' : \\x' -x\\ <S\\x\\} and 
CV{x, S) = {x' eW^ : \x'i-Xi\< S\xi\ for i = 1, m}. 

There are two reasons why researchers study condition numbers. First, 
when we input a real number xi in a computer, the computer can never store 
Xi with 100% accuracy. Instead, an approximate value x[ will be stored. How 
accurate we can store a number depends on the data type chosen for storing 
the number xi. Suppose we store this number with the data type double. 
The relative error 

|a;'^ — 1 , / 1 

< — , which means x e CV I x, 



\xi\ - 252' " y^, 

So, if WCCi(x) = 4, we can ensure 

\h{x')-Mx)\ ^ 1 _ 1 

\Mx)\ - 252 250- 

Similarly, if SCCi(a;) = 4, we can expect 

\Mx')-hix)\ _ 1 _ 1 

\fl{x)\ 252 250- 

The second reason of studying condition numbers is related to the stability 
of algorithms. Even if we can stored the input x accurately, we still cannot 
find the output f{x) with 100% accuracy. It is because errors appear and 
accumulate after every operation (addition, subtraction and etc.) done in 
a computer. How accurate we can compute f{x) depends on the algorithm 
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applied. We say that an algorithm is backweird stable if the computed 
output /' satisfy the following. 



/' = f{x') for some x, s.t. - x|| < ||a;|| 0{e^^hme), 

where ^machine IS the upper bound for the relative error occuring after one 
operation done. As a result, applying a backward stable algorithm, one can 
ensure the computed solution /' satisfy the following. 



11/(^)11 

Or one can expect 

II/' -/(^) 



ll/(^) 



< WNC(x)0(£„,achine)- 



^machine ) • 



Unless specified, log(a;) refer to the logarithm with base 2. (log \x\ — 
log \x' — x\) is called the precision of x'. Roughly speaking, it is the number 
of trustablc (or accurate) bits. logWNC(a;) is called the Worst-case Norm- 
wise Loss of Precision since log WNC(x) 

= lim sup (log|a:| -log|a:'-x|) - (log|/(a:)| -log|/(x') 

— lim sup Precision of input x' — Precision of output f{x'). 

Similarly, log WCCj(a;) is called the Worst-case Component-wise Loss of 
Precision. Besides, we define Stochastic Norm- wise Loss of Precision 
and Stochastic Component-wise Loss of Precision as follows. 

SNLP(x) = lim E logMi^^T^^fM and 
SCLP.(x) = lim E log^M^lziM. 

S-*0 x'^CV{x,S) S\fj{x)\ 

In short, both condition numbers and Loss of Precision are numbers 
telling us how trustable is the computed output when there is round-off 
errors. If these numbers are large, the output is not accurate and we should 
not trust the oTitput. Otherwise, the computed output should be accurate 
and we can trust it. 
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The main goal of this paper is compare the worst-case condition numbers 
with the stochastic condition numbers, i.e. 



WNC(x) Vs SNC(x) , WCC(x) Vs SCC(x), 

logWNC(x) Vs SNLP(x) and logWCC(x) Vs SCLP(x). 

The theorem [1] below is one of our main results. It compares SNC(a;) and 
WNC(x). Denote by e the base of the logarithm of In(-). 

Theorem 1. For any general computational problem with input x G M"^ and 
output y G M", let k = mm{m,n} then 



1 <m^<JJ- ™, 



m - WNC(x) - V m + 2 
- loge < SNLP(a:) - logWNC(x) < 1^8^-^^ + 2) 



Similar results can be found in [5] and [6j. In this paragraph, we explain 
the differences between our theorem [1] and results in other papers. We write 
X ~ N{fi, S) if X follows the multivariate normal distribution with mean 
fi and variance-covariance matrix S. In [B], the following quality (which is 
different from our definitions of condition numbers) was studied. 

E ||[V/i(x),...,V/„(x)r(x'-x)||, 

(x' —x)^N{0,1j) 

where S can be any variance-covariance matrix and Vfj{x) is the gradient 
of fj{x). So, their results are completely different from ours and depend on 
E. The above quantity was studied since, by Taylor expansion, when x' is 
close to X, 

||[V/i(x),...,V/„(x)r(x'-x)|| ^ 

The results in [5] hold only for the problem of solving system of linear equa- 
tions. Our theorem [T] holds for general computational problems. Besides, the 
output /(x) in [S] was considered to be a real number in M. In this paper, 
the output is considered as a real vector in M". 

Similar to theorem [1] above, the corollary [1] below compares SNC(a;) and 
WNC(a;). Comparing with theorem [1], corollary [1] is less explicit and less 
general (only holds when n = 1). But, it provides equality result instead 
of inequality. Both theorem [T] and corollary [T] will be proved in section [21 
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Corollary 1. For any general computational problem with input x G M"^ and 
output ?/ G M, 



SNC(x) ^ 1 ^/"^ 
™C(x) j G::^)-;:;^,-!, il) ^frmseven 



and 



SNLP(x) -logWNC(a;) [-±-^-...-1-1 if mis odd 



_ m-2 ■■■ 3 

loee 1 — — ^ — ... — i — In 2 if m is even 



The theorem [2] below compares WCC(x) and SCC(a;). 

Theorem 2. For any general computational problem with input x G M"* and 
output y G M", if m > 1, 

-log(m- 1) log 3 (^^^^)iQgg ^ SCLPj(x)-logWCCj(x) < -1 and 



2 2 

g-(i+e™.) SCC,-(x) 1 , 2 + 21nm 

< „ , . < - where - — 



v/3(m - 1) WCC,(x) - 2 v/^^T^ 

Note: It can be easily shown that, SCLPj(a;) = logWCCj(a;) — 1 and 
SCCj(x) = 0.5 X WCCj(x) when m = 1. 

Similar results can also be found in [5] . Just like our theorem [21 the 
result in [5] depends on a constant e'^ which approach to as m ^ oo. But, 
the speed of convergency was not discussed. Let alone a formula computing 
the value (or bound) of e'^ for general m. Our theorem [2] only depends on 
the size of the input and output (m and n). Once again, in [5], only one 
problem (solving system of linear equations) was considered. In this paper, 
we consider general computational problems. Theorem [2] will be proved in 
section [31 

Note: In practice, ^^^y^ is not very large (1^^|^ = 20 when the number of 
data input m = 1.0995... x 10^^). So, from the theorems above, we claim 
that the value difference between worst case and stochastic loss of precision 
are normally very small in practice. 

2 Proof of Corollary [1] and Theorem [1] 

For any c G M™ and r G M, let the ball centered at c and with radius r be 
5*^(0, r) = {u G W ■ \u - c|| < r} 
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and let the sphere be 

S^'-^Cr) = {ueW': \\u - c\\ = r}. 
Lemma 1. If u B"^{0, 1), then 

Tf) IT) — 1 

E{\\u\\) = EiWuf) = and E{ln\\u\\) = —. 

m + 1 m + 2 m 

Proof. Since u ~ 5™ (0,1), 

^ , ,,, ,, , The volume of 5"^ (0, r) ^ 

Prob M < r = — = ^ 

^" " ^ The volume of 5™(0, 1) 

So, the p.d.f. (probability density function) of is 

d 



mr 



m—l 



dr 

By the definition of expectation and integration by parts, 

•1 



m + 1 

Ei\\u\?) = I mr'^+^dr 







m + 2 



/■i _i 
E(ln||M||) = / mr"'~Mn(r)rfr = — . 

Jo ^ 



For any vectors u,v E W^/{0}, let the angle between u and v be 



T 
U V 



□ 



•diu^v) = arccos ( - — n~n~~n" ^ [O?^]- 

V ll^ll IpII / 

Suppose u is fixed and || ~ 5'"^~^(0, 1). From [2j, the p.d.f. (probability 
density function) of ■(9(u, v) is 

f^iu,v){t) = \ ^ ^ , , where /^(T) = / (sin(t))'"cit. 

^m-2[T^) Jo 



By integration by part, it can be shown that 



It is easy to check that 

Combining equations ([T]) and ([2]) , for m > 2, 

'7r\ I I (™~^)(™~^)-^ ] if m is odd 



1 (f ) .fm. seven 

Lemma 2. Supposeu is a fixed vector in andv/\\v\\ ~ S'™'^^(0, 1). Then, 
for m > 3, E(| cos'i9('U, f )p) = 1/m, 

if m is odd 

Ei\cosHu,v)\) = { )r-::r:<... ' . and 



E(ln I cos^{u, v)\) 



( (m— 2)(m- 


-4). 


..1 


l (m— l)(m- 


-3). 


..2 


(m— 2){m- 


-4). 


..2 


l (m— l){m- 


-3). 


..1 


1 


1 




m— 2 


m— 


4 


1 


1 




m-2 


m— 
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Y ) (^) zs even 



I — 1 if m is odd 

I — In 2 if m is even 



Proof. By the definition of Expectation, 

r r fsinft))™~2 

E{\cosi^{u,v)\) = / |cos(t)|/^(„,,)(t)rft= / |cos(t) r r i ^ 



/m,-2(7r 

^sin(t))™-2 (sin(t))"-2 



Im-2{Tr/2) Jq im-2(7r/2) 

1 



(m - l)/„_2(7r/2) 
By equation ([3]), for m > 3, 



E(| cos^{u,v)\) 



(m.-2)(m-4)...l 
(ni-l)(m-3)...2 



if m is odd 



(m-2)(m-4)...2 \ / 2 \ -r 

7 — — 5^-7 ( - it m IS even 

(m— l)(m,— 3)...l / Vtt/ 
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Similarly. 



r(l-(sin(t)f)^ 
Jo -'i 





sin(t))'"-2 , 
^ dt 



2W 

^ (sin(t))— 2 - (sin(t))- Im-2{n) - I^{ti) 



-dt 



Im-2{T^) Im-2{T^) 

/m-2(7r/2) - /^(7r/2) _ ^ m - 1 _ 1 
/»n-2(7i"/2) m m' 

The second equality above is due to equation ([1]). Besides, let 

J^(T) = /" (sin(t))™ln I cos(t)| dt. 
Jo 

Then, 

J„ (-) = / ' (sin(t))'" In cos{t)dt = - I (sin(t))™-^ In cos(t)c/(cos(t)) 

7r/2 

cos(t)rf((sin(t))™-Mncos(t)) 

7r/2 

-(sin(t))" + (m- l)(cos(t))2(sin(t))™-2incos(i)^i 









= -/™(7r/2) + (m - 1) J„_2(7r/2) - (m - 1) J„(7r/2). 

Combining the above and equation ([T]), 

J^(7r/2) _ J^-2(7r/2) 1 
/„(7r/2) /„_2(7r/2) m' 

Besides, it is easy to check that 



(4) 



Combining equations ([2]), (jlj) and (I5l), we have 

J„(7r/2) r - ^ - ... - i - 1 if m is odd 

1 1 



/^(7r/2) [ -- - - ... - 2 - ln2 if m is even 

-2(^2) 
-2(7r/2) • 



The proof is completed since E(ln | cos'(9(m, v)\) = ^(^2) • '-' 
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2.1 Proof of corollary [T] 

Proof. Denote by V/(a;) the gradient of /. By Taylor expansion, 

fix') = fix) + ix' - x)^Vfix) + Oi\\x' - xf ). (6) 
Combining the definitions of WNC(x) and Vix, 6) and equation IQ, 

'5-o,,epg,5) S\fix)\ \fix)\ 
Combining the definitions of SNC(a;) and Vix,S) and equation ([6]), 

SNC(x)=hm E (^^^^^^^^) (8) 
= l|V/(x)||hm E (^ IK-^ll><|cos^(y/(x),x--x)| y 



Since x' ~ 5) = Ball(x, by lemma[T], 

, m6 \\x 

E a; — a; = 

x'r~.v{x,5) m + 1 

By lemma [2], 



E \x'-x\ = —. (10) 



if m is odd 



(m-2)(m-4)...l 

" ^^"''^ I ( U-i)U-3)...i ) if ^ IS ^^^^ 



Combining equations ([9]), ffTOl) . ffTTj) and ([7]), we have 



bNC(x) _ I I (m+l)(m-l)...2 



if m is odd 



WNC(x) /'"yrVj m if mis even 

I \ (m+l)(m— / Vtt/ 

Similarly, applying lemmas [T] and [2], it can be shown that 

SNLP(a;) - In WNC(x) _ [ ^ - ■■■ - \- ^ if m is odd 

Ini ~ \ -i^-^-...-i-ln2 if mis even 



□ 



2.2 Proof of Theorem [T] 

Proof. Denote by fj{x) the jth entry of f{x). Denote by Vfj{x) the gradient 
of fj. By Taylor Expansion, 

f,ix') = f,{x) + {x' - xfVf.ix) + Oi\\x' - xf ). 

Let G = [Vfiix), V/2(x), V/„(a;)] G M^x". So, 

f{x') = f{x) + G^{x' -x) + 0{\\x' - sf ). (12) 

Combining the definitions of WNC(x) and V{x,6) and equation f|T2l) . 

WNC x = hm sup " ^1 „, ,1, ^" = ' '' V| • 13 
^-o,,ep(^,5) 511 /(x) II ||/(x)|| 

Combining the definition of SNC(a;) and equation (|T2|) . 

SNC(x) = hm E . ^ 

Let ?7Dy be the singular value decomposition of , i.e. U,V E 
orthogonal matrices, D G M"^™- is a diagonal matrix with entries cxi, ct2, ...cxfc 
on its diagonal where k = min{m, n}, 

G^ = UDV and HGH = (Ti > aa > ...fffe > 0. (15) 

Since ?7 is orthogonal, by equations (IT^ and (IT^ . 

...T^/ N , ^ -a;)||\ , ^ /||DV(x' -x)|| 

SNC X = lim E ^ ^1,;, Ml = lim E " ... ^" 

x'^pcx.^) \ 5||/(a;)|| / 5^0x'r^vix,s) \ 5||/(a;)|| 

Let x" = V[x' -x). By the definition of P(x, 5), x" ~ 5™(0, (5||x||). So, 

/ \\Dx"\\ \ 

SNC(x) = lim E '' ^ ' . (16) 



/ ||Gnx--x)|| \ _ ^^^^ 



5-»Oa;"~B™(0,<5||a;||) V^ll/(^)ll/ ' 

<5-.Oz"~B™(0,51|x||) V 52||J(2,-)||2 ; 
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Let V be the vector in M*" with the first k entries equal to 1 and elsewhere. 
SNC{xf < a^lim E (19) 

= al V f lim E ^ , " 20 

2 II II 2 II II 2 

^ ^ ' By lemmas [Hand H (21) 



(m + 2)||/(x)P 
Since 0"! = HGH and ||f || = ^/k, by equations ( fT3l) and (El 



SNC(x) ^ / A; 



WNC(x) - V m + 2 
Since log(-) is concave function, 

SNLP(x) -logWNC(x) < log Mm + 2) 
On the other hand, by equation (fT6|) 



SNLP(2;) > lim E log 



aix"i 



Let ei be the vector in M*" with the first entry to 1 and elsewhere. 



(Ti||x"|| I cos6'(ei, 



SNLP(x) > lim E log 

<5-^Ox"~B™(0,<5||x||) 

= 1™ ^ log \ ni X/ Ml 

By equation (fT3|) and lemmas [1] and O 

SNLP(x) - log WNC(a;) / " ^^ib " •- " I " ^ if m is odd 



loge \ - ^ - ... - i - ln2 if m is even 

Since Inm = the area of the region {{x,y) : l<x<n,0<y< 1/x}, 

SNLP(x) -logWNC(x) ^ r _iinm-l if m is odd 

loge ~ \ — |(lnm — ln2) — I — ln2 if m is even 

1 , 

> — mm — 1. 
2 

SNC(x) 1 . , , ^ . , . 

TTT^T^/ X ^ — 1= smce log(-j IS a concave function. 
WNC(x) ey/m 

□ 
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3 Proof of Theorem [2 



We write Z ~ iV(0, 1) if Z is random variable following standard normal 
distribution. Below is the well-known Berry-Esseen theorem (See [5]). 

Theorem 3. Let Ui,...,Um be i.i.d. random variables with E(mi) = 0, 
E(m^) = a and E(|mi|^) = p < oo. Then, for any real number a, 



Prob ( a < 



Ui + ... + Ur^ 



a\/m 



Prob(a < Z) 



< 



cp 



where Z ~ A^(0, 1) and c is a universal constant (independent ofm). 

Calculated values of the constant c have decreased markedly over the 
years, from 7.59 (Esseen's original bound) to 0.7975 in 1972 (by P. van Beeck). 
The best current bound is 0.7655 (by I. S. Shiganov in 1986). The lemma 
below follows Berry-Esseen theorem immediately. 

Lemma 3. Let Z ~ A^(0, 1) and ui,...,Um ~ 1] be i.i.d. random vari- 
ables. Then, for any real number a. 



Prob a < 



Ui + ... + Ur, 



Prob(a < Z) 



< 



1 



m 



Proof. Since Um ~[— l,l],P = l/4 and = 1/3. By Berry-Esseen theorem, 



Prob a < 



Ui + ... + Ur. 



Prob(a < Z) 



< 



3^-^c 0.9555c 



A\/m 



m 



□ 



Lemma 4. Let 6,b be positive numbers, s.t. h> 1. Let Z ~ A^(0, 1). Then 



z + 5 



z-5 



dz > 0. 



6 In 6+ / Prob(Z > z)\n 
Jo 

Proof. Let 

F{S) = [ iz + S)\n\z + 5\-{z-S)\n\z-5\dProh{Z > z). 
Jo 



12 



Since Prob(Z > 0) = 1/2, by integration by parts, 

F{6) = [{b + 6) In \b + 6\-{b- 6) In \b - 6\]Proh{Z > b) 

-5\yi5- [ Proh{Z > z)d[{z + S)\n\z + 5\-{z-5)\n\z-5\]. 
Jo 

Since {b + 6) ln\b + 6\ = 26ln\b + 6\ + {b-6) ln\b + 6\ > {b-6) ln\b-6\, F{6) 

> -6\n6- [ Proh{Z> z)d[{z + 6) In \z + 6\-{z -6) In \z-6\] (22) 
Jo 



z + 6 



z-6 



dz. 



= -6\n6- Prob(Z > z)\n 
Jo 

Obviously, F(0) = 0. Besides, 

dF{S) d{z + S) In \z + S\ d{z-S)ln\z-S\ 



(23) 



dS 



d6 



dS 



dProh{Z > z) 



[ ln\z + 5\-ln\z~5\dProh{Z > z) 
Jo 

fz{z)dz<0 since > 1,V^ > 0. 





z + S 


/ In 


Jo 


z-6 



So, F{5) < 0. Together with equation fl23|) . the proof is completed. 



□ 



Lemma 5. Let 6 < VSm be a positive number. Let ui, ...,Um ~ [—1, 1] be 
i.i.d. random variables. Then 



E 



Ui + ... + Ur. 



'm/?> 



5 In 



Ml + ... + Um 



'm/?> 



+ 5 



> 



S 



Proof. Let W = {ui + ... + Urr^^fzfm and fwiy^) be the probability density 
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function of W. By definition of expectation, 



E[{W + 5) \n\W + 6\] 



/3m 



{w + S) In \w + S\f\Y{u))dw 



/3m 



{w + S) In \w + S\fw{'w)dw 



/3m 
/3m 



/3m 



{w + S) In \w + 6\fw{'w)dw 



{w + S) In \w + S\fw{'w)dw 



(5 — w) In \6 — w\fw{,—'uj)dw 



3m 



/3m 



[{w + S) In \ w + S\ ~ {w — 6) In \w — 6\] fw{w)dw 



/3m 



{w + 5) In |iy + (5| - (w - 5) In \w - 5\ dY'voh{W > w). 

Note that: when w = 0, P{W > w) = 0.5 and (w + 6) In |w + 5| = —{w — 
S) In \w-6\ = 6\n6. So, by integration by parts, E[(iy + (5) In + -51n5 

/•v'3m 

= / Pvoh{W > w) d[{w + 5)ln\w + 5\ - (w - 5)\n\w - 5\] 
Jo 

/•V3m 

= / Prob(W^ > w) In 



w + 5 



w — 5 



Since \w + S\ > \w - 5\ for all w > 0, by lemma El E[{W + S)\n\W + S\] 



> SlnS+ Prob(Z > ;z) In 



z + 5 



> 



/3m 



In 



w + 6 



w — 6 



z-5 
dw by lemma m 



/3m 



In 



w + 5 



w — 5 



dw 
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So, -v^ E[{W + S)\n\W + 5\] 



< 





w + 6 


dw = 


/ In 


Jo 


w — 6 


Jo 



/ \n{w + 5)dw — \n\w — 5\ dw 

Jo Jo 



\w\nw — w] 



/3m +(5 



In (5 — w) dw — 



/3m 



In {w — 6) dw 



[wlnw- w]f^+^ + [w\nw-wfs-[w^^w- w]f^~^ 



^ + 6) \n{V3rn + 6) - {V3rn - 6) ln(V3rn-5) -25 In (5 

^3m + 5 
^3m — (5 

Since Inx < x - 1 for all x > 0, E[(iy + 5) In |iy + 5]] 



26 ln(y3^ + 6) + {VS^ - 6) In 



- 25 In 5. 



'Sm + 6 



'3m — 5 

251n(V3m + 6) + 26 - 26ln6 = 25 ( In ( 1 



1 1 -251n5 



6 



+ 1 



□ 



Lemma 6. If ui, ...Um+i ~ [— 1, 1] o'^e i.i.d., 

E(ln \ui + ... + = E((mi + ... + + 1) In \ui + ... + + 1|) - 1 

Proof. For any fixed a G M, 

/I /-a+l 
ln|a + M|(iM= / In |f Ifit; (let f = a + m) 
■1 Ja-l 



(24) 



= [vlnlt;! = (a + l)ln|a + 1| - (a - l)ln|a - 1| - 2. (25) 
So, by the definition of expectation, E(ln \ui + ... + Um+i\) 



J ■■■j \ri\ui + ...+Um+l\dUm+l- ■ -dUi 

j {Ui + ... + Mm + 1) In \ui + ... +Um + 1 M^m " " ' dui 

(mi + ... + - 1) In \ui + ... + '«„- l\dum ■ ■ ■ dui 
■ dui by equation (1251) . 



2m+l 
1 

2m,+l 
1 

2m+l 
1 



-1 J -1 



2m+l 



2dUr 
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So, E(ln \ui + ... + Um+i\) + 1 



-E((mi + ... +Um + l) In \Ui + ... +Um + l\) 



... +Um- I) In \Ui + ... +Ur, 

Since u and —u follow the same distribution, 

E[(mi + ...+ - 1) In \ui + ... + Um+l - 1|] 

= E[{-Ui - ... - U-m+l - 1) In I - Ml - ... - U-m+l - 1|] 
= -E[{Ui + ... + Um+l + 1) In \Ui + ... + Um+l + 

Combining equations and (1501) . the proof is completed. 

In this section, we follow the definition of given in theorem [21 

2 + 21nm 



i/m — 1 

Corollary 2. If Ui, ...,Um+i ~ 1] ^^^^e z.z.c?. i/ien 

, IN In m In 3 
E(ln|Mi + ... + > — ^-1 

Proof. Let = (mi + ... + Um)^/^Jrn. 

E((mi + ... + M„ + 1) In |mi + ... +Um + 1|) 

= E 



^m+1- 



W'-,/— + llhi 



H'./f + 1 



E 



E 



W + 



In 



m 
3" 



W 



In 



> 



ln(m/3) [m 



ln(m/3) 



E{W + 



W + 
3"\ 2 



(26) 
(27) 



(28) 
(29) 
(30) 

□ 



m 



m 
3" 

ln(m/3) [m ^ , 



(In (m + 1) + 1) by lemma [S] 



(In (m + 1) + 1) Since E{W) = 0. 



2 a/"^ 
Applying lemma O the proof is completed. 



□ 
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Denote by e„ the vector in with all entries 1. For any a G M™, denote 
by ||a||i = |ai| + ... + \am\ the 1— norm of a. Below is a lemma from [5], 

Lemma 7. For any increasing function ip : M — M, a G M*" and 6 G R, s.t. 
||a||i = m, 

Prob(|a^M| > 6) > Prob(|e^?i| > 6) and E((^(|a'^M|) > E((^(|e^M|). 
3.1 Proof of Theorem H 

Proof. Denote by fj{x) the jth entry of f{x). Denote by Vfj{x) the gradient 
of fj. By Taylor Expansion, 

f,ix') = f,{x) + {x' - xfVf.ix) + Oi\\x' - xf). (31) 

Combining the definition of WCC(x) and equation ([SI]), 

WCCj(x) = hm sup ^ ^' 



For i = 1, m, let 



gt = Xi X the ith component of V fj{x) 
Then, by the definition of CV{x, 6), 



WCC,,.) ^ ^^..p^^^^g.^. (32) 



Combining the definition of SCC(x), CV{x,6) and (? and equation ([3T 



SCC,(x) = lim E 



{x' -xfVfjix) 



5f,{x) 



E 

n~[-l,l]'" 



T 

u'g 



(33) 



= E by equation dMD. (34) 

Obviously, 

E \u^g\ = E \uigi + ... +Umgm\ (35) 

ue[-i,i]"" mg[-i,i]™ 

< E (|mi5'i| + ••• + l^mfi'ml) (36) 
Me[-i,i]'" 

= l^il E + + E \um\ (37) 

= 0.5(|(?i| + ... + |^?^|) = 0.5||(?||i. (38) 
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Combining equations (IMI) and (138 



SCCj(x) 
' WCCj(x) 



< |. Since log(-) is a concave 



function, SCLF j{x) — log Cj{x) < —1. Combining the definition of SCLPj(x) 
CV{x,6) and (7 and equation fl^ . 



SCLP^(x) 



= In 

> In 

> In 



loge 
ll^lli 



lim E In 



(x' ~xfVfj{x) 



5f){x) 



E 
E 



In 



T ^9 



u 



\9\\i 



m\fj{x)\ «~hi,i]" 
\\g\\i ln(m — 1 

TIM 2~" 



In \u^e\ 



In 3 



by lemma [7] 
1 — Em by corollary [2] 



InWCCfa;) - 



ln(m — 1) In 3 



-l-er 



2 2 

log(m— 1) 



by equation fl32l) . 



That is, SCLPj(x) > logWCCj(a;) - ^ 
log(-) is a concave function, SCC,(x) > y^^2iM=( 

^ ]\ J ^3(m-l) 



i^l^ - (1 + £„)loge. Since 



-(l+Em) 



□ 
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